Method and system for auxiliary data detection and delivery

ABSTRACT

A method and system for auxiliary data detection. Data is received on a hand-held device from a display of a signal on a monitor. The signal is visually displayed on the monitor at a frequency. A modulated video signal is received on the hand-held device from a display device. The modulated video signal contains auxiliary data modulated within a video signal in a substantially invisible way. The modulated video signal is demodulated to obtain the auxiliary data. At least one of the data or the auxiliary data is processed to identify a promotional opportunity available through the hand-held device. The user of the hand-held device is notified of availability of the promotional opportunity through the hand-held device.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.14/797,511, filed on 13 Jul. 2015 (the “'511 Application”), which is acontinuation-in-part of U.S. application Ser. No. 14/444,493, filed on28 Jul. 2014 (the “'493 Application”), which is a continuation of U.S.application Ser. No. 13/770,394 (now U.S. Pat. No. 8,850,468), filed on19 Feb. 2013 (the “'394 Application”), which is a continuation of U.S.application Ser. No. 12/979,834 (now U.S. Pat. No. 8,453,171), filed on28 Dec. 2010 (the “'834 Application”), which is a continuation of U.S.application Ser. No. 12/511,809 (now U.S. Pat. No. 7,886,329), filed on29 Jul. 2009 (the “'129 Patent”), which is a continuation of U.S.application Ser. No. 11/686,156 (now U.S. Pat. No. 7,590,992), filed on14 Mar. 2007 (the “'992 Patent”), which is a continuation of U.S.application Ser. No. 09/829,223 (now U.S. Pat. No. 7,213,254), filed on9 Apr. 2001 (the “'254 Patent”), which claims priority to U.S.Provisional Application Ser. No. 60/195,542, filed 7 Apr. 2000 (the“'542 Application”), U.S. Provisional Application Ser. No. 60/207,460,filed 25 May 2000 (the “'460 Application”), and U.S. ProvisionalApplication Ser. No. 60/266,238, filed 4 Feb. 2001 (the “'238Application”). The entire disclosures of the above patents and patentapplications (e.g., the '511 Application, the '493 Application, the '394Application, the '834 Application, the '329 Patent, the '992 Patent, the'254 Patent, the '542 Application, the '460 Application, and the '238Application) are incorporated by reference.

BACKGROUND

The present invention relates to interactive hand-held devices, and moreparticularly to methods and apparatuses for receiving and decodingmodulated signals for use by hand-held devices, and receiving benefitsfrom receptions of the signals.

The hand-held device of the present invention receives modulated signalsfor purposes including enjoyment, promotion, transfer of information,data collection, commercial verification, security, education, andtransactions or verifications at points of sale, as well as othercommercial, personal, entertainment, or amusement purposes collectivelyreferred to herein as “promotional opportunities.” Data is preferablysent to the hand-held device by optical, audio, and/or electricaltechniques, although the device can be independent of a specifictransmission protocol and therefore may alternatively be sent via othermethods. Data may be received by the hand-held device by utilizing asleeve, cradle, or docking station; through an optical lens, by use of aPCMCIA or alternate computer port, by use of an audio pickup device(such as a microphone), or by FM, AM, and/or other radio frequencytechniques, or other techniques as may be required based on the type oftransmission means selected. Use of the device may allow users toreceive, process, and/or store promotional opportunities.

U.S. Pat. No. 4,807,031 to Broughton et al. (“Broughton”), titled“Interactive Video Method and Apparatus,” generally relates to in-bandvideo broadcasting of commands and other encoded information tointeractive devices. The invention described therein relates generallyto interactive educational and entertainment systems, and is describedin one embodiment in the context of television program control of toyslocated where there is a television receiver, as within a residence.

To encode control data capable of providing a benefit to a user,Broughton discloses a novel method of luminance or chrominancemodulation of a video signal that creates a composite video signal,whereby control data is created by modulating the video signal. Thenovel modulation method alternately raises and lowers theluminance/chrominance of adjacent horizontal scan lines to create avideo subcarrier that contains the control data.

Under Broughton, the video signal is not being replaced with other data,nor is the data being added as a separate signal along with the videosignal, rather, the video signal itself is modulated to subsequentlycreate the control data. Therefore, the control data is a part of, orcontained within, the video signal. The encoding method also includespreview and remove circuitry to ensure suitability or the presence ofdata encoding and removal of data encoding, respectively.

The control data is transmitted either by television broadcast means, orby prerecorded video players that are connected to a video display. Thecontrol data is then received by the video display where a video fieldof the video display is modulated by control data. The control data isthen detected with either opto-electronic or RF (radio frequency)detection means that discriminate the program material from the controldata. The detected control data is further reproduced such that thecontrol data can be used with an interactive device.

A practical example of a device as described above is thecommercially-sold hand-held game device for receiving and detecting suchcontrol data called the “Wheel of Fortune” ITV Play-Along Game, intendedto be used while viewing a television program presentation of the famoustelevision show of the same name. The device, produced under license bythe assignee of Broughton, was a palm-sized device that included aphotosensor within its case to receive video signals. The device, uponreceiving composite video signals, then discriminated the control datafrom the video program material and caused a liquid crystal display(“LCD”) on the face panel of the toy to present portions of a wordpuzzle. Thereby users of the device may play the game along with acontestant, or play in response to a videotaped presentation of thegame. The “Wheel of Fortune” interactive television (ITV) game togetherwith its hand-held control device including keyboard was commerciallyavailable in 1988.

An improvement on the method of modulation described in Broughton isdescribed in U.S. Pat. No. 6,094,228 to Ciardullo et al. (“Ciardullo”).In Ciardullo, improved methods of modulation are disclosed. Control datais inserted on the visual portion of a video signal by changing theluminance of paired lines in opposite directions, thus allowing largeramounts of data to be modulated in a signal. Broughton and Ciardullo areboth incorporated herein by reference.

Efforts by others to provide hand-held devices capable of receivingtransmission of modulated data from a video display are represented byU.S. Pat. Nos. 5,594,493; 5,761,601; 5,767,896; 5,907,350; and5,953,047. Of these, U.S. Pat. No. 5,907,350 discloses a method forstoring data on a so-called smart card, which is contended to receive,decode and store encoded data signals comprising redeemable coupons saidto be embedded within television segments and transmitted along withnormal television segments. The device of U.S. Pat. No. 5,907,350 is ahand-held unit that receives luminance signals from the televisiondisplay in accordance with the principles of Broughton. The receivedvideo signals are decoded and stored within the card for future use. AnLCD readout enables Universal Price Codes (“UPC”) corresponding to thestored data. A scanner reads the UPC codes at a redemption site, and thestored coupon is then erased from a memory of the card. A microprocessorchannels the decoding and storage aspects, and a keypad allows use andinput.

The term “smart card” as used in the above patents, connotes ahand-held, portable device, not conceptually different from theabove-mentioned “Wheel of Fortune” ITV game device. However, the termdoes not only apply to those patents.

As a generic term, “smart card” gradually has come to mean a card thatlooks like a credit card but includes a microchip or microprocessorembedded or incorporated into the card. The smart card may be referredto as a “fingerheld” computer, typically including a data storage mediaranging from less than a kilobyte up to a megabyte (if not more), andare said to have originated in France. See, for example, Ognibene, P.J., “Card Smarts,” Technology Decisions (July, 1999). Smart cards may,according to a line of reference, also be called “chip cards.”

Prior efforts by the inventor of this patent application include UnitedStates Utility patent application entitled “Interactive Optical Cardsand Other Hand-Held Devices with Increased Connectivity”, U.S. Pat. No.7,296,282, filed 21 Jan. 2000, by Edward J. Koplar and Daniel A.Ciardullo (“Koplar”), which is incorporated by reference herein. Koplarrelates to various methods and apparatuses for use with promotionalopportunities, such as interactive advertising and gaming. Koplardescribes various methods for receiving and providing data to hand-helddevices, as well as apparatuses for use with promotional opportunitiesand methods of using the same.

While the hand-held devices of the inventive subject matter describedherein may be in the form of a smart card, they may also be in the formof other hand-held devices such as mobile phones. Recently, mobilephones have become equipped with increased capacity to store and processinformation, and many phones now offer limited network or Internetaccess. Industry experts estimate that by 2002, more than 100 millionmobile phones will have Internet access in some fashion and that by2003, more than 1 billion mobile phones will be in use worldwide. Mobilephones, with or without Internet access, may become powerful tools whenappropriately configured to receive auxiliary data.

The personal digital assistants (“PDAs”) are another form of hand-helddevice that may provide users with promotional opportunities under thepresent invention. The most popular manufacturer, Palm Computing,introduced its first PDA called the “Palm Pilot” in 1996. The latestversion of the Palm Pilot, Palm VII, was introduced in 1999 and was thefirst PDA to include wireless Internet access without needing peripheraldevices such as a modem.

PDAs and mobile phones, while capable of providing Internet access, asof the time of this invention have limited capabilities, including slowtransfer rates of data. It is desirable to provide users of suchhand-held devices and similar devices such as smart cards with hand-helddevices that are compact in size yet rich in content that encourageusers to participate in various promotional opportunities without havingprolonged delays in receiving and processing information. Smart cards,mobile phones, PDAs, and similar hand-held devices are all capable ofparticipating in the promotional opportunities described in this patentapplication.

BRIEF SUMMARY

For purposes of the inventive subject matter described herein, the terms“hand-held device” and “card” mean an interactive device of portablecharacter, preferably of hand-held type which may be carried in the palmby a user, between fingers of the user, or is otherwise intended to beeasily grasped and handled manually by the user, including creditcard-like devices.

The term “sponsor” is used herein in its broadest possible sense, andmay include without limitation entities that issue hand-held devices andentities that accept them or provide redemption services for users ofthe devices. Sponsors may also include health care and medicalinstitutions and other service or eleemosynary organizations.

The term “computer” is used herein in its broadest possible sense, andmay include without limitation a laptop or personal computer, mobilephone, personal digital assistant, or other computer-like device.

Among the several objects, features, and advantages of the inventivesubject matter may be noted the provision of interactive hand-helddevices for carrying out various modes of novel and commerciallyadvantageous signaling, information-transferring, and value-indicatingmethods. The hand-held devices include portable devices such as smartcards, mobile phones, and personal digital assistants (“PDAs”), whichcontain hardware circuitry to receive auxiliary data by various possibletransmission protocols. The hand-held devices, to provide some of thepromotional opportunities described herein, preferably have circuitry toconnect to the Internet, which may be referred to herein interchangeablyas “network access”, “wireless access”, “Internet access” or “wirelessInternet”. These hand-held devices react from the reception of auxiliarydata from sources including radio transmissions, computer systems, videorecordings, video transmissions or other sources for reproduction ofvideo, audio and/or other data. The hand-held devices, preferablyutilize the transmission methods described in Broughton and Ciardullopertaining to reception of auxiliary data from a video display viaelectrical and/or optical detection and decoding means. All videodisplays, whether a television set or monitor, television-like, orimage-presenting display device capable of displaying an imagecontaining a modulated video signal, may in the present specification becollectively referred to simply as “display devices.” The hand-helddevices may also respond to auxiliary data received from non-videodisplays, such as may be received directly from audio transmissions or adecoding box.

Signals are received, detected, and reproduced by the new hand-helddevices for various promotional opportunities including: enjoyment;promotion; coupon or prize validation; advertising by sponsors;advertising verification and polling; transfer of information; datacollection; commercial verification; security and access; education;game playing; transactions, verifications, or redemption by sponsoringentities or related commercial locations at points of sale including theInternet; or other commercial and non-commercial purposes.

Auxiliary data transmitted to the hand-held devices are encoded using asystem of modulation, preferably as described in Broughton or Ciardullo,but alternatively by other known methods of modulation in the artincluding through the use of the vertical-blanking interval (VBI), audiosubcarrier, electromagnetic subcarrier, and RF detection of videodecoding. The video signals containing the auxiliary data may betransmitted to a display device by means of a broadcast, cable, standarddigital video methods, satellite or home video.

Other objects, features, and/or advantages of the inventive subjectmatter include:

A method of interactive advertisement and promotion signal transmissionusing interactive hand-held devices in such a manner that both the modeof transmission of information and the display of received informationand/or value indicative thereof will attract viewers' attention,stimulate viewers to pay attention to a transmission (which may becommercial advertising), and effectively entice viewers to respond andtake action in response to its reception, as well as achieving theforegoing usage purposes;

A method of such signal transmission that is cost effective, brandefficient, and fun for users of the new interactive hand-held devices;

A method of such signal transmission using interactive hand-held deviceswhich effectively increases customer awareness, retention, anddifferentiation of advertisers' messages and products;

A method of such signal transmission using video interactive hand-helddevices which allows viewers of television (or video) presentations,including advertising, to directly interact with the devices (such astelevision receiver or display);

A method of such signal transmission using interactive hand-held devicesthat builds store (point-of-sale) traffic, fosters consumer loyalty, andgives viewers a reason to pay more attention to a company's products,services and advertising;

A method of such signal transmission using video interactive hand-helddevices which allows viewers of television (or video) presentations,including advertising, to upload received information to a computerwhich displays and uses relevant Internet addresses, textual informationor code, and other promotional information usable by a computer orcomputer-like device; and

Hand-held devices which are intrinsically simple, effective andeconomical to make and distribute widely, and are reliable and easy touse.

The following are merely illustrative of some of the advantages andobjects which the new system provides: television advertising responsedetermination; interactive advertising and promotions; attraction ofviewers' attention; effectively increase consumer awareness andretention of commercial advertising, messages, announcements,promotions, and specific products and services; increased customerdifferentiation of products and services; stimulation of viewers towatch commercials; increase of store traffic in response to commercialmessages; fostering of consumer loyalty; enhancement of viewerinvolvement in program content, including commercials; enhancement ofviewer retention of the content of commercial and other messages;enhancement of the value of commercial messages; increasedproduct/service sales; saving of advertising costs; acceleration ofresponse time of customers following delivery of commercial messages;verification of contests and awards; enhancement of viewer retention ofthe related website domain names; reduction of barriers related toe-commerce opportunities; additional and sometimes instant rewards andinformation obtainable via the Internet. The uses and advantages aremore fully developed in the following description.

Briefly, herein is described a system which uses various hand-helddevices to derive benefits from the reception of auxiliary data.Auxiliary data is encoded by modulation of a signal, such as videoand/or audio, which thereby creates a composite video signal includingauxiliary data and continuous video program data. The composite videosignal is transmitted to and displayed by a display device so that itmay be received and used by a hand-held device.

When the hand-held device is a smart card, the auxiliary data isreceived from composite video signals by optically detecting andextracting the auxiliary data directly onto the smart card. A user firstpositions the optical receiver of the smart card so that it is capableof receiving video signals from a display device. The smart card, whenactuated and appropriately positioned, begins receiving video signals.Circuitry on the smart card discriminates whether auxiliary data ispresent in the video signals (i.e., where the video signals received arecomposite video signals). If auxiliary data is present, the hand-helddevice reproduces the auxiliary data for its use.

One method of receiving auxiliary data from composite video signals whenthe hand-held device is a mobile phone or PDA includes detecting andelectronically extracting the auxiliary data from the video signal byuse of a decoder box. The decoder box preferably receives a video signalfrom its source through the use of RCA jacks. The decoder box thenprocesses the composite video signal to determine whether auxiliary datais present. The auxiliary data, when present, is transmitted from thedecoder box to the hand-held device. Transmissions to the hand-helddevice can occur through RF or other audio techniques, so that thehand-held device need not be aimed at or physically connected to thedecoder box. For example, the auxiliary data may be encoded within theaudio of the composite signal and/or the composite signal may includeaudio that triggers (e.g., causes or otherwise directs) the hand-helddevice to obtain the auxiliary data (e.g., by downloading the auxiliarydata via the Internet or another network). Although the opticaltransmission method is preferable for smart cards and the electricaltransmission method is preferred for mobile phones and PDAs, bothmethods as well as other methods known in the art including through theuse of vertical-blanking interval (VBI), audio subcarrier,electromagnetic subcarrier, and RF detection of video decoding, all ofwhich are interchangeable and may be used with any of the hand-helddevices of the present invention.

Among the several methods herein described may be noted a method ofvideo interactive advertising and promotion, comprising normaltelevision programming content modulated to contain auxiliary data,where the auxiliary data is associated with a sponsor and carriesinformation of special significance relative to the sponsor and ofspecial value to television viewers, distributing interactive devices ofhand-held character to television viewers for their use and convenience,the devices being capable of selectively receiving auxiliary data orreceiving and decoding composite video signals, providing incentives forviewers to use the interactive devices for obtaining the auxiliary databy rewarding the viewers with information of special value, theinteractive devices being capable of retaining indication of havingreceived the information, and providing redemption of the special valuefor users at a redemption site where users may present the interactivedevices.

When auxiliary data is reproduced by use by the hand-held device,various signals, indications, display readouts, or other interactiveevents provide the user with a benefit according to content of theauxiliary data. The various interactive events described in Koplar,incorporated by reference herein, are usable interchangeably by and inconjunction with the hand-held devices and methods of using same withthe present invention. The interchangeability includes selective use ofthe features of the present invention, along with selective use of anyof the various apparatuses and methods of Koplar.

The designs of the hand-held devices are typically in the shape and formof smart cards, palm devices, or mobile phones. Additionally, otherhand-manipulable devices of a similar size and nature that are capableof receiving and processing auxiliary data as described herein are alsowithin the scope of the present invention and may receive data byelectrical or optical methods as may be appropriate or desirable. Suchhand-manipulable devices include stuffed animals or toys, snapshot-typecameras, replica sports helmets, scaled racing cars, or replica baseballbats.

Devices and methods are provided herein to allow an electronic coupon tobe redeemed at a point of sale (“POS”) using existing point of salehardware and software. Additionally, coupons may be configured to beused only once and with a reasonably high degree of security.Furthermore, check out time is not seriously degraded and multiplecoupons may be redeemed at the same POS with relative ease. One methodof using a hand-held device at a POS involves the user giving the deviceto the cashier. The cashier then locates the correct coupon orredemption value on the hand-held device using a scrolling function incombination with the LCD. The hand-held device is then held such thatthe LCD is facing towards the cashier and a photodetector faces the barcode scanner. The “redeem” button is then pressed, and the photodetectoron the hand-held device passes over the bar code scanner. The timer inthe hand-held device resets the device to non-redeem mode if a scan isnot detected within thirty seconds. When the scanning laser light isdetected by the hand-held device, the device beeps and a numeric code isdisplayed which may correspond to a stock keeping unit (SKU) number orother information regarding receipt of product shipment, inventory,and/or stocking. Other codes, including an additional numeric code suchas the serial number of the hand-held device or a security code may alsobe displayed. The cashier then enters the numeric code or codes and theappropriate discount or value is applied to the purchase of theconsumer. The timer may also delete the coupon if the hand-held deviceis scrolled to the next coupon, or if the device is powered down andthen powered up again. The coupon may also be erased eitherautomatically or manually by pressing predetermined buttons on thehand-held device.

The methods of data transmission as described in Broughton and Ciardulloare more complicated when data is being sent to hand-held devices viacomputer monitors, as opposed to other display devices such astelevisions. Computers constantly refresh the screen displayed by themonitor. Therefore, modulating the video signal displayed by a monitorin real time is difficult and often impractical as a result of theconstant refreshing of the video card and factors relating to thedisplaying of video by the computer. One or more embodiments of theinventive subject matter described herein transmit data from the monitorto the hand-held device in a less obtrusive manner.

Various other apparatuses and methods of the inventive subject matterdescribed herein are set forth in the following description and claims.Similarly, other objects and features will be apparent or are pointedout more particular herein below.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a perspective view schematically illustrating use of any ofseveral embodiments of an interactive card in accordance with andembodying one example of the present invention.

FIG. 2 is a perspective external view of an embodiment of an interactivecard in accordance with and embodying one example of the presentinvention.

FIG. 3 is a perspective external view of an embodiment of an interactivecard in accordance with and embodying one example of the presentinvention.

FIG. 4 is a perspective external view of an embodiment of an interactivecard in accordance with and embodying one example of the presentinvention.

FIG. 5 is a perspective view of internal features of the interactivedevice of FIG. 6 of the embodiment of FIG. 8 showing an interactive cardof one example of the invention.

FIG. 6 is a perspective external view of another embodiment of aninteractive card of one example of the invention.

FIG. 7 is a perspective view of internal features of the interactivedevice of FIG. 6 of the embodiment of FIG. 6 showing an interactive cardof one example of the invention.

FIG. 8 is a perspective external view of another embodiment of aninteractive card of one example of the invention.

FIG. 9 is a schematic diagram of circuitry that may be used to providevarious embodiments having circuit features evident in this diagram.

FIG. 10 is a schematic diagram of other circuitry that may be used toprovide various other embodiments.

FIG. 11 is a perspective view of a docking station without a hand-helddevice.

FIG. 12 is a perspective view of a docking station with a hand-helddevice.

FIG. 13 is a perspective view of a decoding box.

FIG. 14 is a front view of yet another embodiment of an interactive cardof one example of the invention, and referred to as a sports card.

FIG. 15 is a perspective view of yet another embodiment of aninteractive card of one example of the invention, and referred to as asports card.

FIG. 16 is a block diagram of other circuitry that may be used toprovide various other embodiments.

FIG. 17 is a block diagram of other circuitry that may be used toprovide various other embodiments.

FIG. 18 is a block diagram of other circuitry that may be used toprovide various other embodiments.

FIG. 19 is an electronic/optical signal detection circuit.

FIG. 20 is a block diagram of the method of optical detection from amonitor by a hand-held device.

FIG. 21 illustrates another example of the inventive subject matter

Corresponding reference characters identify corresponding elementsthroughout the several views of the drawings.

DETAILED DESCRIPTION

Referring to FIG. 1, in a system of one example of the invention, and inaccordance with the various methods herein described, composite videosignals are received and projected by a display device 10, to bereceived by a first embodiment of an interactive hand-held device of theinvention 12. The composite video signals may include a video signaland/or an audio signal. Display device 10 is representative of atelevision screen, video monitor or other video display, movie screen,computer monitor, video-converted display or video-like display, capableof receiving analog or digital video or video-representative signalsfrom a suitable source, such as a television transmitter, a videotape, astreaming video server, a Digital Versatile Disc (“DVD”), or thecomputerized display representation of such a source of image content.For present purposes, it will be assumed that display device 10 is akinescope or other conventional type of television display or monitor(which may, of course, include multiple or single-beam types ofprojector displays).

Display device 10 may schematically represent a video display fordisplaying video signals but may also be any sort of electron gun,active, array or passive array display device capable of providing notonly imaged information in a visible mode but also auxiliary information(e.g., data) in a substantially transparent mode. Display device 10 maybe further characterized as a computer monitor or display, as well as aportion or computer window of display device 10. Display device 10 mayalso be a high definition or digital television, or other digital videopresentation device. Display device 10 may vary in size, and may besmall like a Sony Watchman™, or large like a movie screen or a SonyJumbotron™. Video signals receivable by display device 10 include thosedelivered by microwave relay, satellite retransmission or cable,streaming and other types of downloadable or viewable computer videopresentations, and those generally made available by wired or wirelessmethods.

Hand-held device 12 may be in the form of any type of hand manipulabledevice such as a smart card, cell phone, PDA, or other palm like device.Enclosure 14 of hand-held device 12 is illustrated in FIG. 1 as takingthe form of a smart card. Hand-held device 12 may be held in the palm orbetween the fingers of a user in the generally vicinity of displaydevice 10, typically within the same room and, when necessary, orientedso that front surface 16 faces display device 10, and thereby hand-helddevice 12 may optically receive light from a visual representation ofthe video signal from display device 10. When hand-held device 12 is asmart card and configured to optically receive auxiliary data, it istypically is within the same room and suitably oriented so that device10 may receive light from a visual representation of the video signalfrom the display device as shown in FIG. 1.

Hand-held device 12 is depicted in FIG. 2 as a PDA and shown in a mannerin which it may be held and used. Hand-held device 12 may include aPDA-sized housing 14 that encloses various circuits and circuitcomponents (“circuitry”) of the device. Housing 14 may be of anyhand-held shape desired that is functional and hand manipulable,examples of which include a PDA (as shown), cell phone, PCMCIA card,snapshot-type camera, replica sports helmet, replica baseball bat orfootball, or scaled racing car.

A brand name of a sponsor 17, and possibly information or indicia thatmay induce a user to associate device 12 with sponsor 17, may beimprinted on front surface 16 of hand-held device 12 above LCD 44. Whenhand-held devices 12 are inexpensive to manufacture, such as when theyare in the form of smart cards, sponsors 17 may be found with higherfrequency. However, when hand-held devices 12 are a user's personaldevice such as a cell phone or PDA that have primary uses other thanreceiving promotional opportunities, such indicia may not be lessfrequently included. Sponsor 17 may be a company providing the hand-helddevice 12 to a user of its service, such as a hotel, phone company, PDAservice, or place of business, that uses device 12 for access,privileges, and/or rewards. Sponsors 17 may also include variousentities such as advertisers, Internet websites, television shows, otheroccurrences, programs or events, for which use of hand-held device 12will provide means for a user to obtain promotional opportunities.

Hand-held device 12 is illustrated in FIG. 2 to have both RF antennae 68and lens 36 so that device 12 may receive auxiliary data by both opticaland electrical (e.g., wireless) techniques. When hand-held device 12contains such reception means, it may be used with various promotionalopportunities in various locations. Thus, with one example of thepresent invention, a user may receive auxiliary data on hand-held device12 at a game inside a stadium or arena electrically via RF antennae 68,and may also receive auxiliary data while watching a game at homeoptically by positioning front surface 18 towards display device 10 (asshown in FIG. 1). Users may thus participate in promotional in multiplelocations using the same hand-held device 12. Optionally, the hand-helddevice 12 can audibly receive the auxiliary data encoded in audio of thecomposite signal and/or the hand-held device 12 can wirelessly receivethe auxiliary data via the antennae 68 in response to receiving theaudio signal (or a portion thereof).

Button 22 of hand-held device 12 can provide means for selecting betweenreceiving auxiliary data via electrical means (such as through RFantenna 68) or by optical means (such as through lens 36). The user,based on their location, can select the means by which to receive theauxiliary data, thereby saving power and preventing user from receivingundesirable auxiliary data. However, hand-held device 12 may beconfigured to monitor the data input means to determine whetherauxiliary data is being transmitted by any of the methods describedherein.

As shown in FIG. 3, a leading edge 18 of hand-held device 12 isoutfitted with lens 36 to provide means for receiving auxiliary data.Visible from front surface 16 of hand-held device 12 is LCD 44 and fourlight-admitting diodes or other electro luminescent light sources 20 a,20 b, 20 c, and 20 d. Depending on its desired promotional use,hand-held device 12 may include both LCD 44 and lights 20 a, 20 b, 20 c,and 20 d, one of the foregoing, or neither. Hand-held device 12 may havevarious users controls as described in Koplar, which may include akeypad or touch screen (not shown) to allow user input for specificfunctions and additional uses.

Hand-held device 12 is depicted as a smart card in FIG. 4 in a manner inwhich it may be held and used. Hand-held device 12 includes a creditcard or wallet-card-sized housing 14 that encloses the circuitry ofdevice 12.

A leading edge 18 of hand-held device 12 may include a lens or smallaperture 36 to admit video signals to a photosensor 30 within housing14, as is described herein. Visible from front surface 16 of hand-helddevice 12 are a plurality of light-emitting devices 20 which may belight-admitting diodes (LEDs) or other electro luminescent lightsources, including, for example, regions of a LCD of either active orpassive type.

A typical manner in which hand-held device 12 is used to opticallyreceive auxiliary data is as follows: A user watches a televisionpresentation such as a commercial, television program, movie, or othervideo-displayed program. When such a presentation is displayed, the usermay then hold hand-held device 12 device as shown in FIG. 3 or 4,depending on the type of device 12 used. The user then positionshand-held device 12 so that leading edge 18 of device 12 will bepositioned for receiving video signals from display device 10. Hand-helddevice 12 or display device 10 may have indicia informing the viewer topress the pressure membrane or button 22 at a preferred time, such aswhen a commercial of a sponsoring entity appears. Although the term“button” is used herein, it is considered to include membrane switches,pressure or touch regions capable of activation and carrying out thefunction of button 22. When the user presses button 22, the video signalwill be detected and received by a photodetector (the process to bedescribed below) within housing 14. The composite video signal comprisesa video signal modulated with auxiliary data, in accordance with theteaching of the above-referenced Broughton and/or Ciardullo each ofwhich are herein incorporated by reference. A discriminator, implementedby circuitry determines whether auxiliary data has been transmittedalong with the video signal of the presentation.

A typical manner in which hand-held device 12 is used to electricallyreceive auxiliary data is as follows: video signals are sent to andreceived by a decoder box 82 as shown in FIG. 13. Decoder box 82discriminates the auxiliary data and transfers it to hand-held device 12by RF or by other means, such as IR, a computer port, or direct wireconnection. Decoder box 82 may otherwise receive the data through othermethods such as through use of a super VHS, fire wire, or coaxial jack,or may be connected to an RF antennae, integrated into the circuitry ofa display device, or otherwise connected so that the decoder boxreceives the video stream. As may be observed in FIG. 13, decoder box 82contains antenna 84 so that it may transmit RF signals between it andthe hand-held device, RCA jacks 86, and a network interface jack 88 sothat it may be connected with a computer network or directly to acomputer.

An appropriately outfitted decoder box 84 may receive transmissions fromother means including IR and direct connection through a computerinterface such as a PCMCIA. Decoder box 84 may also function as atwo-way device, so that it may receive feedback from the hand-helddevice. The decoder box may, with a graphics rendering unit such as aScorpion processor, display information received from the hand-helddevice on the display device. The decoder box may contain networkaccess, which may supplement or replace the need for network access onthe hand-held device. The decoder box may also contain means to controlperipheral devices, such as a VCR. Users, upon receipt of informationsuch as a television schedule on their hand-held device, may transmit arequest to the decoder box or VCR to schedule the recording of atelevision program.

Alternatively, the received video signals may be sent through aninterface of hand-held device 12 to a computer or computer-like devicewhere the discrimination of auxiliary data may be implemented in thecomputer's hardware or software. The computer interface apparatus andmethod of using same is described in Koplar.

If the discriminator determines that the transmitted video signalscontain auxiliary data or auxiliary data is received, the circuitry ofhand-held device 12 may then cause an audio transducer 34 (as shown inFIG. 7) present on the printed circuit board to sound a tone or “beep”,providing assurance that the auxiliary data has been receivedsatisfactorily. A suitable aperture may be provided on the front or rearsurface of housing 14 to allow sound to pass from audio transducer 34.

Hand-held device 12 may contain more advanced means of providing soundto a user. Hand-held device 12 may contain advanced sound circuitry toenable device 12 to play digital audio, including popular audio formatsMP3 and MIDI. The sound is preferably preset on the hand-held device 12.However, sound may also be downloaded to or prerecorded on hand-helddevice 12 and configured such that when an event occurs for which asound is desired, the event triggers the circuitry of device 12, whichcauses a sound to be transmitted through audio transducer 34 or otherhardware (not shown) designed to play sounds. Hand-held device 12 mayalso be configured to play sounds upon user demand, such as when useractuates button 22.

Hand-held device 12 may also be configured to initially prohibit userfrom taking advantage of prerecorded or preset sounds or promotionalopportunities so that sounds or promotional opportunities areunavailable to the user. The hand-held device 12, upon receivingspecific auxiliary data which directs device 12 to unlock the sounds andpromotional opportunities so that the opportunities are available to theuser. The sounds and promotional opportunities may then be available tothe user for a limited time or indefinitely.

An additional feature may include the hand-held devices 12 coordinatingtheir sounds. Once the sounds are available for use on hand-held device12, users may perform the sounds as per the desired promotional event.Through use of the computer interface port as described in Koplar, orother communication means such as IR or RF, a hand-held device 12 maysynchronize songs with other hand-held devices 12 and other peripheraldevices by transferring and receiving information and triggers from thevarious devices.

The circuitry may also cause operation of one or more of lights 20 a, 20b, 20 c, and/or 20 d of FIG. 2 or FIG. 3, to indicate a value that hasbeen received. By outfitting hand-held device 12 with lights 20 a, 20 b,20 c, and/or 20 d, device 12 is capable of providing promotionalopportunities in the methods described in Koplar.

FIG. 6 illustrates features of hand-held device 12, including theprovision of aiming light 24 on front surface 16 of device 12. Aiminglight 24 is typically included on embodiments of hand-held device 12where video signals are optically received by device 12, such as toindicate that device 12 is positioned such that it is capable ofcapturing video signals emanating from a display device 10. When button22 is pressed while the leading edge 18 of hand-held device 12 isgenerally oriented toward display device 10, received composite videosignals on device 12 cause illumination of aiming light 24 to indicatethat video signals are being received. Aiming light 24 illuminates forso long as data is being received, for a predetermined period of timesufficient to indicate that hand-held device 12 is oriented or aimedcorrectly, or for a long enough time period as to enable the encodedsignals to be received by device 12. Aiming light 24 may be included onembodiments of hand-held device 12 where the video signals areelectrically received to alert the user that auxiliary data is beingreceived, such as through IR, RF, or the computer interface.

Referring to FIG. 7, the internal features of hand-held device 12 areillustrated. The features may be seen to include a printed circuit board26 having integrated circuits 28 including push-button 22 and aiminglight 24. Circuit board 26 is shown to include photocell 30. Photocell30 is oriented so that it may receive light through a suitable aperture36 (not shown) along leading edge 18 of hand-held device 12, whichaperture 36 may be at locations as shown in FIGS. 2 and 3. Circuit board26 is powered by one or more batteries 32 or other suitable cells (notshown), intended to provide sufficient power for operation of hand-helddevice 12 for days, weeks, months, or years. Batteries 32 may be usedwith other similar sources of portable power, such as solar cells 33 (asshown in FIG. 9). It will be appreciated in the art that the electronicsof a PDA, cell phone, or other computer-like device capable ofprocessing auxiliary data as described herein may contain a significantamount of additional circuitry to provide functionality unrelated to thepresent invention.

Three lights 20 are shown present on circuit board 26, and they may takethe form of LEDs. However, as will be understood from study of FIG. 16,circuit board 26 may include the capability for more than three suchlights. For example, it may include circuits for energizing up to nineLEDs.

Referring now to FIG. 8, an embodiment of hand-held device 12 is shownto include LCD 44. LCDs 44 capable of presenting the graphic content inmonochrome or color are preferably used. However, LCD 44 may also be atextual display consisting of 16 digits, but less or more displaycapability may be provided based on usage and need.

Hand-held device 12 has power button 38 which when depressed willinitiate operation of LCD 44 to display a coupon, a prize notification,or other information indicating receipt by device 12 of auxiliary data.For example, LCD 44 may be used to display a first coupon in acircularly linked list of offers. Power button 38 may also held toinitiate receiving video signals, or may be used for other display orcontrol purposes separately from button 22. Scroll button 40 allows auser to traverse, i.e. scroll, information retained or received byhand-held device 12, for example, the circularly linked list of offersreceived and stored on device 12. It will be appreciated in the art thatany of the various buttons 22, 38, 40, or 42 may be implemented by usingother “pre-existing” buttons (i.e., with other functionality) onhand-held device 12, such as the numbered buttons on a cell phone orshortcut buttons on the PDA. It will furthermore be appreciated thatthese buttons 22, 38, 40, or 42 may be implemented via touch screen,such that physical buttons 22, 38, 40, or 42 may be implemented“virtually” on hand-held device 12. Expiration button 42 acts in themanners disclosed in Koplar. Offers received by hand-held device 12 maybe indicated not just by illuminating lights 20, but by textualinformation and graphics displayed on LCD 44 as may be observed in FIG.6.

FIG. 5 shows circuit features of the card embodiment of hand-held device12 without housing 14. LCD 44 is evident, as are controls 38, 40 and 42,and a pair of batteries 32 located on the right side of the circuitboard.

FIG. 9 shows a chart of an alternate embodiment of hand-held device 12having an input-output means preferably in the form of a PersonalComputer Memory Card International Association (“PCMCIA”) interface,such as may commonly be found on a laptop computer. PCMCIA interface 70of hand-held device 12 connects with PCMCIA connector 72 of a computeror computer-like device to transmit information back and forth.Connections with PCMCIA interface 70 to a computer are preferably madeby a PCMCIA port, but the connection means may also be other knowncomputer and computer-like slots, connections, and ports such asEthernet, Token Ring, infrared (“IR”), RF, Small Computer SystemInterface (“SCSI”), Universal Serial Bus (“USB”), parallel port(“Parallel”), serial port (“Serial”), IEEE 1394 FireWire (“FireWire”),S/PDIF, AES/EBU, fiber optical cable, and Apple Desktop Bus (“ADB”).Hand-held device 12 may also utilize portable data storage, such asflash memory, mini-disc, or stick memory, to manually transferinformation from hand-held device 12 to a computer. The types ofconnections used in various applications of hand-held device 12 may varybased on factors including cost, transfer speed, acceptance by industry,and user preference. Implementations of hand-held device 12incorporating PCMCIA interface 70 and their methods of use are disclosedin Koplar and may be interchangeably used with one or more embodimentsof the present invention.

FIG. 10 is another embodiment of the hand-held device 12 with aninput/output interface in the form of a PCMCIA interface 70 and dockingstation 76 as shown. The docking station 76 may receive discriminatedauxiliary data or composite video signals, and transfer either tohand-held device 12 by PCMCIA interface 70. Alternatively, as shown inFIG. 11, docking station 76 may contain photocell 30 so that it may beused to optically detect the video signal from a display device 10. Theneither the entire video signal may be transferred to hand-held device 12so that device 12 can discriminate the auxiliary data, or the dockingstation 76 can discriminate the auxiliary data and transmit theauxiliary data to device 12. The data may then be transmitted tohand-held device 12 by IR, RF, or by PCMCIA interface 70. FIG. 12illustrates a manner in which hand-held device 12 may receive auxiliarydata from docking station 76.

Referring back to FIG. 10, docking station 76 may charge battery 22 ofhand-held device 12 by its battery charging control 80. Docking station76 may implement various hand-held device 12 functions and advanceddevice 12 functions by controlling microprocessor 50 through interfacedrive 78.

Turning to FIG. 15, a version of hand-held device 12 which may be termeda “sports” device 12S is shown. It includes housing 14 which is providedwith front surface 16 which may include a visual field 54 for bearingthe image of a sports contestant. In addition, LCD 44 is providedtogether with button 22 which allows a user to initiate operation ofhand-held device 12, preferably with an RF antenna 68 oriented forreceiving RF signals, or alternatively for optically receiving compositevideo signals through photocell 30 (as shown in FIG. 2), such as byholding hand-held device 12 so that leading edge 18 is generally towardsdisplay device 10 presenting a program relating to the sports celebrityor other person depicted on display device 10. Sports device 12S may beoutfitted, configured, and used in the manners described in Koplar andherein.

For example, as a celebrity batter takes his turn during a baseballgame, television viewers direct their sports device 12S at displaydevice 10 to receive composite video signals, or preferably configurestheir device so that they may electrically receive the auxiliary data.In both instances, viewers receive real time statistics about theirfavorite athlete during the telecast via sports device 12S.Additionally, the users of sports device 12S at a live sporting eventmay receive auxiliary information without the use of display device 10,such as by RF or IR. Thus, sports fans may get live statistics andtrivia through use of their hand-held device 12 while they watch thegame live and at home. Fans with obstructed views or sports fanatics mayobtain additional information and have fun while enjoying a livesporting event. The information received by sports device 12S may takemany forms including, as noted, statistics such as batting average, homeruns, or other personalized information about the player that is storedin device 12 for future reference or updating. An example of suchpresentation is shown in FIG. 14. The stored information may thenconveniently displayed on LCD 44 for the device owner by the push ofscroll button 40 or update button 56, or otherwise transferred through aPCMCIA interface 70 to a computer.

Use of sports device 12S in connection with a baseball game is merelyillustrative. Other sports in which players have statistics which changeduring play, over a year, or which may include information unique to theplayer, such as hometown or previous teams, may similarly be downloadedor conveyed by device 12S. Sports device 12S may also be used inconnection with concerts and other special presentations. Users mayacquire useful information about the concert or special presentation,such as the names of performers, set lists, the venue location andhistory, and related Internet addresses through receipt of auxiliarydata on device 12. Persons with hearing impairments may receive realtime text of live speeches on hand-held device 12. Sports device 12S maybe used to receive information relating to monuments, buildings, cars,animals, etc. during the course of a live or video presentation, whereinthe transmission to device 12 contains auxiliary data.

Hand-held device 12 may also be used in trivia applications as follows:The user watches a trivia game show, such as the television show “WhoWants to a Millionaire?” The user may play along by utilizing hand-helddevice 12. The question is downloaded and received by hand-held device12, either during the broadcast or prior to the broadcast, by use ofauxiliary data and the methods described herein. The questions aresynched to display device 10, such that when display device 10 firstpresents the questions, the auxiliary data triggers hand-held device 12to display the question and possible answers. The user chooses thecorrect answer, or places the answer in order in a timely manner, ormust otherwise comply with the rules of trivia game request in a similarmanner to that of the contestants. The user receives points or creditfor the correct or timely answer, based on various factors which mayinclude the number of other at home users who answer correctly or speedof answer. The percentage of at home viewers either nationwide, local,or by other area may be displayed in real or delayed time either ondisplay device 10, Internet website, or on hand-held device 12. Usersmay achieve regional or national rankings based on their performance.The final results from the various games may then be stored at a centraldatabase. Users who score high may receive recognition, a prize, or anoption to play in a future televised game.

Hand-held device 12 may also be used other trivia applications asfollows: User takes hand-held device 12 and appropriately configures itto receive modulated video from display device 10. During an event, suchas the “Grammys”, performers appear on the screen and perform a song.The user is then notified that a promotional opportunity is available,either through display device 10 or hand-held device 12. This preferablyoccurs as modulated video triggers a graphic rendering unit in decoderbox 82, creating an icon to appear on display device 10. (Thereby notappearing on display device 10 of people who do not have such a unit.)Hand-held device 12 provides the user with an opportunity to purchase asong or album on which the performers appear. The money for the purchasemay be made through online automation or a third-party Internet website,or may be entered or taken from a website. Alternatively, if the user sodesires, an order requiring final approval and payment information canbe sent to the user e-mail address. The user may then purchase the musicin the following manners: directly, through a third party's website suchas the affiliate program at “CDNow”, or directly from the manufacturer.Otherwise, a link for the special purchase can be sent to the user'spreferred e-mail address. Bonus or additional rewards may be sent alongwith the purchased product such as a bonus disc with previouslyunreleased material, live songs or videos, or future product discount.The user may choose to have the song sent in appropriate digital formatto their e-mail account, or the selected song may become available for aperiod of time at a website for the user to download. Finally, the songcould become part of a list that, when full to the length of a CD, theCD is automatically created and sent to the user. It will be appreciatedin the art that this method of purchasing may be used with any othersimilarly sold products (such as jewelry on the “QVC network”).

Another example of this technology is as follows: Any participatory gameshows produced for television distribution, whether produced anddistributed for broadcast, cable, direct satellite, or close circuittransmission; whether distributed by means of home video or DVD may beencoded with auxiliary data enabling viewers of programming material todecode either optically or electronically; process, store and displaythe data on hand-held device 12 for the purpose of winning prizes oraccumulating points. Participatory type games include: (1) Gamesrevolving around trivia questions dealing with subjects such as sports,music, news, movies; (2) Games revolving around Multiple choicequestions; (3) Games dealing with true or false/yes or no typequestions; (4) Games dealing with guessing letters; and (5) Gamesdealing with question revolving around Who? What? Where? When? How? orWhy?

The viewer of display device 10 participates remotely either at home, ata public establishment such as a restaurant or bar, or in a hotel room,all by means of any properly outfitted hand held device 12, includingsmart cards, PDA type devices, electronic decoding boxes, cell phone orwireless handset. As the answers are decoded processed and stored inhand-held device 12, the viewer then chooses a response to the questionsby depressing from a choice of buttons the correct answers. In case ofdevices with LCD display 44 such as the PDA, cell telephone, or wirelesshandset, the choice of answers might be display on LCD 44. By touchingthe choice with a stylus, pointer or finger, the answers are pre-definedby the televised show in determining the correct answers. Accumulationof winning answers are tallied within in the remote device and displayedby a numeric tally. Answers may also be in the form of an alphabeticaldisplay.

Point accumulation or winning combinations are then redeemable forvarious levels of prizes, and are redeemable by a program's sponsor atpoint of sale, through the computer or computer interface, or by theprograms fulfillment process.

Thus, in the case of a national televised trivia contest, participantsgo to various sports bars where they are issued a play-along game piece.During the televised game, participants in the various sports barsacross the country compete by guessing the correct answers and tallyingpoints not only against each other, but also the studio contestants onthe shows. The winning cardholder may be awarded a cash prize or otherconsideration including a possible appearance on the television show.The redemption may occur through hot syncing his card to the programsweb site, to a sponsor's retail site, or other means of fulfillment. Itwill be appreciate in the art that this application may be utilized byany of the hand-held device embodiments described herein or with Koplar.

In FIG. 16, a schematic circuit diagram of one form of circuitry thatmay be used to provide the features and functions of the severalembodiments described herein, elements are identified by theirconventional numerology conventional characters proximate of theelements. Optional photosensor 30 is seen to be constituted by aphotodiode or phototransistor Q2 which provides the signal toamplification and filter circuit consisting of an operational amplifierU1A including a circuit consisting of the components of C1, R19, R18 andC12 interconnected between the output and the inverting input ofoperational amplifier U1A, and so providing an amplifier and filter.Thus amplifier and filter signal is provided through R15 to a filter andfrequency shaping circuit comprising of an operation amplifier U1Bhaving in its feedback circuit the components C10, C11 and R16. Thesecircuits provide a filtered, frequency-shaped output through a resistorR5 through a circuit consisting of operational amplifier U1C andresistor R11. Operational amplifier U1C has an output coupled to a diodepair D11 which thus provides a pair of outputs, the upper to a circuitcomprising an operational amplifier U1D, providing in effect a bufferwhose output is a data signal delivered through a resistor R20, and thelower of the diodes D11 providing a signal through a resistor R1 to afurther resistor R7, across which is a capacitor C3. The node betweenresistors R1 and R7 provides 16 kHz signal that is of the horizontalline scan rate or frequency television display optical signals receivedby photosensor 30 (Q2). The horizontal scan frequency of conventionalNTSC color signals is, more precisely, 15.625 kHz. (approximately 16kHz, as here described). Similarly, the discriminator circuitry providesat the output of operational amplifier U1D an 8 kHz signalrepresentative of the auxiliary data received, as produced byalternate-line luminance modulation of the display. The discriminatorcircuit thus described provides both the 16 kHz and 8 kHz (moreprecisely, 7.8175 kHz), which constitutes the actual data, anddetermines whether the 8 kHz data carrying the information (data)content of the received auxiliary data is present by determining whetheraverage signal magnitude of the 16 kHz or 8 kHz signals is greater, sothat if auxiliary data is present, e.g., there is 8 kHz modulationenergy in the received light signals, they are provided to amicroprocessor of the U2 of the circuit for decoding the informationcontent within the 8 kHz signal. Thus, the received composite videosignal is amplified and filtered to de-emphasize the 16 kHz signal andretrieve the 8 kHz signal carrying the data. For example, wave shapingand frequency shaping filter circuits provided by operational amplifiersU1A & U1B de-emphasize the horizontal retrace frequency and amplify thehalf-frame rate carrying the data content of interest. The circuitsfilter the frame rate signal and use it as a reference to compare itsrelative signal magnitude against the 8 kHz data-carrying signal. If theretrieved 8 kHz signal is of greater magnitude than the conditioned 16kHz signal, auxiliary data is present and may be utilized by theprocessor. Microprocessor U2 is preferably of commercially availabletype Z86E02, and operates according to machine instructions stored inits resident ROM memory. Flowcharts of the functions performed by themachine instructions are set forth in the appendix. Although notseparately shown microprocessor U8 includes random-access memory (RAM orDRAM) in which decoded data is stored. Thus, microprocessor U2 is todecode and store, or store indication of, the data present in the 8 kHzsignal, as by illuminating one of more of the LEDs and causing them toremain on as evidence or indication of the data, substantially inaccordance with the teachings of Broughton, and to drive displays of thedevice and/or its LEDs or other signal devices, as well as to provideany audio function, as well as also to energize an aiming LED7 whenauxiliary data is present. So also, if desired, a signal may be drivenby U1 in response to the presence of the auxiliary data, thus providingan aiming light function indicating, in simple effect, that thehand-held device is receiving composite video signals.

For providing illumination of lights 20 or 20 a-20 d described above inconnection with the various embodiments, shown here are nine LEDsD9-D18. It will be understood that not all of these LEDs need be usedfor a specific embodiment of hand-held device 12. That is, the circuitmay be configured so as to drive one, two or three or more of the LEDsas according to the desired function and use of the device.

Shown adjacent microprocessor U2 are two diodes D15 and D20 which areused to bypass any noise generated from the microprocessor. The speakerSPK1 is element 34 shown in FIG. 7. A clock crystal XTAL1 provides 8 MHzclock signal to microprocessor U2. Capacitors C8 and C9 are part of theclock circuit.

Immediately adjacent to the clock circuit is a switch 51 to provide thefunction of button 22 in the several embodiments for initiatingoperation of the device. Switch 51 is tied to a power supply circuitcomprising power cells a resistor 14, V2 and V3 (each of 1.5 vpotential), and a resistor 14, and including a switched analog powercircuit comprising a transistor Q1 and resistors 12 and 13 which enablepower to be provided to the array of LEDs D8-D18.

Referring to FIG. 16, there is shown in FIG. 17 another version ofcircuitry invention is shown which may be used to provide still otherembodiments of the hand-held devices of the inventive subject matter. Itincludes a microprocessor U2 which may of commercially available typeZ86X08. It is similarly provided with 8 kHz and 16 kHz signals (moreprecisely 15.635 kHz and 7.8175 kHz) which respectively constitute thehorizontal retrace signal and the data signal at half of that frequency,since the auxiliary data encoded on alternate horizontal lines and so iseffectively present at only half the horizontal scan rate, as explainedabove. These signals are identified as VEILA and VEIL REF that areprovided as inputs to microprocessor U2. Therefore, it will beunderstood that operational amplifier circuits consisting of operationalamplifiers U1A, U1B, U1C and U1D provide the same functions ofamplification, filtration, frequency shaping and discrimination, as wellas buffer circuits described in FIG. 16. The microprocessor, clocked bycrystal X1 at 8 MHz, decodes the data provided by the composite videosignal, stores it in internal RAM or DRAM and drives an LCD 44illustrated as circuit component LCD 1 which may be, as shown, a16-character by 2-line display. It may instead be a 12-character by3-line display, for example, or other format appropriate for the displaymessage to be presented at the surface of the device.

Power for the circuit is provided by two cells BT1 and BT2 of 3 voltpotential to a circuit including Q2 to provide not only main power byauxiliary power, the latter being delivered to the microprocessor formaintaining its continued operation, such as for date and time keepingeven when the display is not being operated and collectively referred toas battery 22. A switch S1 is provided for performing the function ofpower button 38 (as shown in FIG. 8) in the several embodiments and,when pressed, enables power to be provided to the LCD1 and otherswitching functions may be provided by switches S2, which controlsscrolling of the LCD 44, and switch S2 may be actuated by scroll button40 shown in FIG. 15. Similarly a switch S3 provides a date function thatmay be actuated by expiration button 42. These signals control inputs tooperational amplifiers U3A and U3D, respectively, and these are providedas inputs to the operational amplifier and also to LCD1. The clearswitch will, when operated as described above clause clearing of thedate, and the scroll switch enables the user to cause the message acrossthe LCD 44 as that in FIG. 8. Operational amplifiers U3C and U3B providevoltages useful according to the connection illustrated.

The circuit embodiments shown are used in conjunction with a method forproducing a composite video signal containing video program material andcontrol data, and displaying the signal on a television and decodingsuch control data for ancillary use, wherein the method includesmodulating at least one video field within the viewing area of atelevision in such manner that the modulation is substantially invisibleto the television viewer, thereby producing a video subcarrier componentof the signal (which is as indicated detectable at an 8 kHz rate)containing the data; and detecting the component to reproduce the datafor the ancillary use. Although luminance modulation is particularlyuseful for that purpose when using hand-held devices in accordance withthe invention, chrominance modulation may instead be employed asaccording to Broughton. Furthermore, the improvements in modulationdescribed in Ciardullo may likewise be used. Various alternativemodulation (and corresponding modulation) schemes may be employed asalternatives, including (but not limited to) amplitude shift keying(ASK), frequency shift keying (FSK) modified frequency shift keying(MFSK), and phase shift keying (PSK). And although the substantially 8kHz data-carrying submodulation is at half the NTSC horizontal scanrate, other submodulation techniques may be used at other multiples orsubmultiples or other periodic rates which are other functions of thehorizontal scan (retrace) rate.

FIG. 18 is a block diagram of circuitry of yet another possibleembodiment of the interactive hand-held devices, designated 12F, whichshows discriminator and microprocessor circuits 50 which are inaccordance with FIG. 16 or 17, and receiving composite video signals byphotosensor 30 so as to decode the auxiliary data and provide eitherlight signals as described above, or, preferably, to drive LCD 44according to the foregoing description. Thus, it includes an update orinitiating button 22, as according to the sports device 12S and whichbegin the process of optical detection. In addition, an FM receiver 62and associated antenna 64 (collectively seen as RF antenna 68 in FIG. 2)may be provided as a surface-mount module added to the printed circuitboard of the device. FM receiver 62 and associated antenna 64 may be ofcommercially available design, suitable for being powered by battery 22or cells (not shown) provided in accordance with the foregoingdescription of various embodiments. Control button 66 may optionallyinitiate operation of FM receiver 62 so that it sends FM-demodulatedsignals to microprocessor circuits 60. The FM receiver 62 can be anytype of RF antenna that is of the size and accordance with the inventivesubject matter, such that the discrimination may take place on adecoder, and the results are sent to the hand-held device 12 via RF.

Hand-held device 12F may be sports device operating otherwise accordingto version 12S described in relation to FIGS. 14 and 15, but theaddition of FM receiver 62 gives the new device the capability of beingused in sports stadiums and other performance venues where video signalsare not available. Thus, the device 12 may receive either opticalsignals or electrical signals. In the case of a sports device accordingto FIGS. 14 and 15, the user may as described above receive statisticsabout a player depicted on the device, or about a player at bat.According to a preferred method of using device 12F, the same auxiliaryinformation as would be transmitted via a television broadcast for beingreceived on a display device 10, there to be displayed as compositevideo signals, is in addition transmitted by low-power FM signals in theFM broadcast band of 88-108 MHz. For this purpose, FM receiver 62 may bepretuned to an appropriate frequency for such low-power signals. The FMsignals may include batter statistics, for example, as batters come tobat, so that holders of the new devices 12F may update their devices astheir preferred batters come to bat.

Hand-held devices 12 may have use in various sports events,entertainment presentation, conventions, museums, tours, and guidedevents at which holders of devices might not conveniently use them whilein the presence of a television display carrying auxiliary data, butwhere the location in which the devices will be used will permitlow-power FM or other RF signals. Other commercially significant usageof the devices of FIG. 16 can be made, similar to the methods of usingthe previously described versions. Similarly, hand-held devices 12 maybe adapted to receive AM signals, or auxiliary light signals such as IR(infrared) light signals, different from the composite video signals.

Similarly, hand-held devices 12, whether or not sized as “cards”, PDAs,or cell phones or other conveniently hand-held size, and whether in theform of racing devices, may simulate other objects, vehicles,personalities, or devices for house-hold, personal or business use, soas to provide a verisimilitude or facsimile representation of suchdevices or usage, so that the hand-held device suggests to the user themanner of its use, or conveys to the users special feeling, sense,identity or association with a field, activity, sport, entertainment orother use of the device it represents. In games, for example, in whichsome or all of the information content useful in the game may beprovided by auxiliary data, hand-held devices of the inventive subjectmatter configured to provide a verisimilitude or facsimilerepresentation of such devices or usage may give players of the gameenhanced sense of play, participation or gaming identity. The foregoingexamples are merely illustrative of the many possibilities.

When used in a system for deriving benefit from encoded data includingmeans for producing video signals modulated with auxiliary data, such asby providing commercially sponsored television programming includingauxiliary data, hand-held devices 12 of the inventive subject matter canhave special impact for television response, advertising and promotions,as the new system and devices are useful in providing, among others, thefollowing commercially advantageous results when they are distributed toviewers and used by viewers to receive auxiliary data transmitted withnormal programming content, which may include advertising, promotionsand other sponsored transmission.

Another feature of the inventive subject matter is the desire to have animproved method of redemption for hand-held devices 12. A method forhand-held device 12 redemption is described in the context of anoptically loaded card 12O. However, such a method could also be employedwith other hand-held devices 12 that have coupon redemptionrequirements. The components of an electronic couponing card 12O whichsupport the method for card 12O and value redemption include:

1.) LCD 44 or other human readable method to display data,

2.) On board computation to sequence the logic defined below,

3.) A method to load “electronic coupon” information onto card 12O,

4.) Buttons 22 or other method (such as a touch screen) to sequencethrough the coupons loaded on card 12O,

5.) Button 22 or other method (such as a touch screen) to put the cardin “redeem” mode, and

6.) Laser detection hardware described below.

Presently, most POS systems use a computerized cash register tied toinventory management software and bar code scanning hardware. Oneembodiment of a method of using an electronic coupon card 12O andexisting hardware and software located at a POS for electronic couponredemption is described in the following steps:

1.) Coupon data is encoded onto a video signal, e.g. television signal.The encoded data also contains additional text describing the couponalong with a numeric code to be entered at the POS.

2.) The user points the multi-use card 12O at the television screen whenthe television signal containing the coupon data is airing. Theelectronic coupon data is then captured by card 12O by opticallyreceiving the data that is presented on the display. The coupon datacould also be captured with methods other than optical detection,including but not limited to radio frequency (RF) or hard-wiredelectronic data transfer.

3.) After capturing the electronic coupon data, the user can then viewcoupon information such as requirements, valid dates, valid POSlocations, etc. on LCD 44 of card 12O. (Other numeric codes described inthe following steps that are used for identification and securitypurposes are not available for viewing by the user at this time).

4.) When the user desires to redeem the electronic coupon, card 12O istransported to a POS and the user buys the desired item or otherwisefollows the requirements of the coupon offer.

5.) At the check out, the cashier takes card 12O and locates the correctcoupon using the scrolling function (usually a button) of the card 12O,in concert with viewing the data on LCD 44.

6.) The cashier then presses the required “redeem” button on the card.The cashier then has approximately one minute to complete steps 7 and 8below. Requiring this button to be pressed reduces the likelihood that acoupon will be accidentally redeemed through bright ambient light.

7.) When the correct coupon is located, the card is held with LCD 44facing the cashier and the back of the card facing the bar code scannerlocated near the cashier. The bar code scanner is the same scanner thatis used for purchases and conventional paper coupon redemption.

8.) When the scanning laser light is detected by card 12O, the cardbeeps and the numeric code described in step #1 is displayed on LCD 44.In addition, another code that represents the serial number of card 12Ois also displayed. The second number displayed may also be a specialcode generated as a combination of the downloaded numeric value (step#1) and a hidden internal (to the card) serial number. The second numbercan be generated by various standard encryption methods and may be usedto increase the security of the transaction because it would uniquelyidentify card 12O and transaction but would not be easily duplicated.

9.) The cashier then has 30 seconds to enter either the numeric codeonly or both the numeric code and card serial number (for greatersecurity). After 30 seconds, card 12O automatically erases the coupon.The coupon may also be erased quickly by a press of the “redeem” or“scroll” keys during the 30 second period. (The scroll key is used toallow multiple coupons to be located and redeemed efficiently asdescribed below).

10.) If additional coupons are to be redeemed at the same time, theabove steps are completed for the first coupon. Additional coupons areredeemed with the cashier repeating steps 5 through 9 above.

In an alternative embodiment, steps 8, 9, and 10 are replaced with thefollowing steps:

8′.) When the scanning laser light is detected by card 12O, card 12Obeeps and displays a numeric code, or alternatively an alphanumericmessage, on LCD 44. In addition, another code that represents the serialnumber of card 12O is also displayed. The second number displayed couldalso be a special code generated as a combination of the downloadednumeric value (step #1) and a hidden internal (to the card) serialnumber. The second number can be generated by various standardencryption methods and could be used to increase the security of thetransaction because it would uniquely identify the card and transactionbut would not be easily duplicated.

9′.) The cashier, upon viewing the number code or alphanumeric message,can determine whether or not the triggered card is the card that waspre-selected to win a prize or to receive a special value or discount.If the card alerts the cashier that a prize has been won, the cashiermay optionally enter the second number (step #8b) into a database toverify that card 12O has not been tampered or altered. The prize mayalso be erased quickly by a press of the “redeem” or “scroll” keysduring the 30 second period. (The scroll key is used to allow multipleprizes to be located and redeemed efficiently as described below).

The following describes how detection of a laser scan across the cardcan be accomplished: Almost all bar code scanning devices use a redlaser of either the helium neon (HeNe) or diode types. The scannersdeliver an extremely bright light in the 832.8 nanometer (nm) wavelengthfor the HeNe or 640 nm wavelength for the diode lasers. The wavelengthspread for a laser is extremely small, which allows for narrow opticalfilters so that non-laser ambient light can be filtered out. A simpleelectronic/optical circuit is shown in FIG. 19 and can be used to detectthe signal.

As shown in FIG. 19, electronic/optical circuit 100 includes opticalfilters 120 a and 120 b, load resistor 140, microprocessor trigger 160,and phototransistor 180.

Optical filters 120 a and 120 b are used to prevent the electronics ofthe card from being triggered by ambient light, e.g. bright sunlight.Optical filter 120 a is a neutral density type to reduce the intensityof light and optical filter 120 b is a red bandpass type.

A photodetector is required to detect the laser light. This detector canbe phototransistor 180 since only an ON/OFF signal is required.Phototransistor 180 has a small resistive load on it, as represented byload resistor 140 so that phototransistor 180 is less sensitive tolight. Load resistor 140 therefore reduces unwanted triggering.

The output of electronic/optical circuit 100 is fed into amicroprocessor on the card where the output then becomes the triggerthat causes the beep, etc. as described in the above steps.

There are complexities when encoding signals in real-time videodisplayed by monitors, such as with computer monitors. As described inCiardullo and Broughton, the substantially invisible signal methodsencode with one video display line brighter and one line dimmerthroughout the entire monitor. Using a constant stream of encoded videocontent with computers requires low-level devices drivers that aredifficult to coordinate with the computer operating system and the videocard that it controls. Monitors, unlike traditional televisions wherethe video content is formatted prior to its reception, generate thevideo content at the operating system level, such as with Microsoft™Windows™, by use of a video card. It is therefore difficult to controlthe display of a computer monitor unless the video data is generatedfrom an application program that makes calls to the operating system todisplay requested data.

The present system utilizes computer software and hardware including theoperating system, application program, video card, and monitor, as wellas a hand-held device to receive data by the user. Under the presentsystem, the application program needs to be executed so that it willbegin to transmit its data. The preferred method to initiate anapplication program is to trigger the start of the application programon specified user interactions with Microsoft™ Internet Explorer′ orsimilar web browsing software. When users interact with their webbrowser by moving the mouse and clicking or double-clicking, the webbrowser can selectively execute the application program on specifiedevents. The application program is stored as a dynamic link library(DLL), such that the web browser can call and initiate the DLL on eventswhere the web browser determines that it needs the DLL file.Installation of the DLL plug-in prior to initiating one or moreembodiments of the present invention may occur so that the web browsercan call the application program. When a user attempts to provideinteractions to their hand-held device for the first time by utilizingthe web browser, the appropriate web site will perform an action thatmay require calling the application program. The user will then beprompted by the web browser for the user's permission to download andinstall the DLL. After installation of the DLL, the action that the webbrowser wanted to fulfill will execute and the desired results willoccur. Alternate arrangements of the present invention may be used, asthe application program may be configured to be a stand-alone program orotherwise integrated within another application program.

The application program is initiated when the user performs variousactions at points throughout the website being explored with the webbrowser. Upon initiation, the application runs and there is an imagedisplayed on the monitor so that the user can capture the data receivedfrom the image on their hand-held device. In one embodiment, the imagewill appear in the title bar of the maximized web browser. The imagemay, however, be located anywhere on the monitor that is visible. Forexample, the image may appear in the eyes of a pumpkin and people withan appropriate hand-held device could thereby capture the opportunity orbenefit on their talking scarecrow doll (hand-held device) thatinteracts with a ghost story. Furthermore, the image is not limited to abox shape; it may be virtually any shape capable of displaying a largeenough image so that users at a specified distance from the monitor cancapture data.

The stream of data received by an optical detector of the hand-helddevice is used to trigger actions on the device or to pass data directlyto the device for its use. Data received by the hand-held device can beused in multiple different ways, including those mentioned herein and inKoplar.

In one method of the present invention, the data received by thehand-held device acts as a trigger initiating action on the hand-helddevice. The device, upon receiving data, looks to see if the data itreceived matches one of its triggers. For example, if the hand-helddevice is in the form of a plush mouse, and the mouse receives a triggerthat initiates a phrase such as “Hello boys and girls”, then uponreceipt of the trigger the device will repeat the specified phase.Triggering the device optimizes the amount of data that needs to bepassed to the hand-held device.

The inventive subject matter described herein includes a mechanism toreceive data on a hand-held device from the computer or the Internetwithout wires. Under one embodiment of the present invention, there isno feedback path from the hand-held device. However, it will beappreciated that the hand-held device may be enhanced with means forfeedback by including RF, IR, USB or other means described in Koplar.The hand-held device may collect data and utilize opportunities receivedfrom data collection by any of the means described herein or in Koplar.

Under one embodiment of the present invention, a communication path isestablished to send data streaming from the monitor to the hand-helddevice. The data is sent by strings of ones and zeros from a varyingvisual image on the monitor. Ones are detected when the image emanatesthe frequency, and zeros are detected when the image emanates nofrequency. Thus, under one embodiment where the image is shown in thetitle bar of a web browser, a solid white color in the title barindicates that no frequency is present and alternating white and blacklines indicate that frequency is present. For each instance in time forwhich the hand-held device is active and its optical detector faces themonitor, if it reads the alternating white and black lines a data bit ofone is recorded, and if the lines have not be ready a data bit of zerois recorded. By varying the appearance and frequency of the display ofthe colors on the screen, data can be transmitted to the hand-helddevice.

A complexity in transmitting the data to the hand-held device under oneembodiment of the present invention is the differing computer, monitor,and video card configurations transmitting the data. Depending upon theconfiguration, the visual image on the monitor will vary to transmit theproper frequency to the hand-held device. Further, the bit per secondtransmission rate of the monitor will vary with the refresh rate. Theinventive subject matter may act independently of the computerconfiguration as will be described in more detail below.

The monitor typically has a CRT tube which draws a picture across thecomputer screen and continues down the length of the monitor from top tobottom before beginning with the next reiteration. The monitor emanatesan image containing a frequency that the device during each successiverefresh (e.g., redrawing) of the screen. During each refresh, an imagewill display on the specified section of the monitor either indicatingthat a frequency was present or was not. The hand-held device looks forthat specific frequency and records a data bit based on its result. Ifthe frequency displayed by the image is 18 kHz, the hand-held device oneach refresh of the monitor will detect whether an 18 kHz image ispresent. If the 18 kHz image is present, the data bit present is a one;otherwise, the data bit present is a zero.

Under one embodiment of the present invention, the device receives abinary data stream and one bit per monitor refresh. If the monitorrefreshes at 60 Hz, there are 60 bits of data transmitted to thehand-held device in one second. At every monitor refresh, the hand-helddevice will look to see if the frequency was present or not. If thefrequency is present the hand-held device records a data bit of one, andif the frequency is not present the device receives a data bit of zero.The binary data stream provides the data, which can be used as a triggeror stored.

An example of the foregoing is as follows: The application program wantsto send a binary code of 1011 to the hand-held device at a frequency of18 kHz. When the monitor draws one field (i.e., one picture on thescreen), it puts the 18 kHz image on the screen. The optical detector ofthe hand-held device reads the monitor, detects the 18 kHz image, andrecords a data bit of 1. On the next refresh, the application programwants to show a data bit zero and so it erases the image on the screenand puts up a white block (e.g., thereby not containing the 18 kHzimage.) When the hand-held device looks again at the monitor, it doesnot detect the 18 kHz image and therefore records a data bit of 0. Thecard thus so far has received data bits of 1 and 0. On the next refresh,the monitor displays the 18 kHz image and the hand-held device reads theimage and records a data bit of 1. On the final refresh, the monitorleaves the 18 kHz image in place and the hand-held device again readsthe image and records another data bit of 1. Thus, the card read thecode from the application.

The data signal is preferably sent multiple times to insure that thehand-held was positioned appropriately and not too much outsideinterference impeded the transmission. The hand-held device may be setto use each data bit it receives, or it may compare the results of thestream it received with the previous stream to ensure that the same databit is not replicated within the device. An event manager on thehand-held device may make determinations as to what happens when data isreceived.

As briefly described above, one embodiment of the present invention actsindependently of the computer configuration. The application programruns independently by varying the image based on the standard horizontalfrequency. The horizontal frequency differs with every monitor setup.The horizontal frequency is determined by 3 factors including the numberof pixels on a line, the number of lines in a picture/screen, and thevertical refresh rate. Therefore, video cards configured of differingresolutions will have different horizontal frequencies. The differinghorizontal frequencies impact the frequency at which the image on themonitor is displayed, and therefore the horizontal frequency is takeninto account by the application program when displaying the image.

For example, if a computer video card is configured to run a monitor in800×600 running at 60 Hz have and the desired image frequency is 18 kHz,the image will have one line set to white, the second is black, thethird line white, and fourth line black, and so on. The result of thatis an image frequency of 18 kHz. If the video card was configured to1024.times.768, the first 11/4 lines white need to be white, the next11/4 lines black and so on to get the 18 kHz frequency.

The application program makes the adjustment based on the horizontalfrequency. Operating systems such as Windows make the number of pixelsper line and the number of lines per screen available. A statisticaltesting program measures the vertical refresh rate by recording the timeit takes for screen drawing to occur. The application program thenstores the configuration in the windows registry. The registry is datastamped so that on a daily basis the application will re-run thisconfiguration program. It will be appreciated in the art that thenstatistical data could however be stored in alternate locations, andthat other variations of time stamp checking could occur.

The hand-held device may be any of the configurations suggested inKoplar or herein, including smart cards, cell phones, PDAs, toys, andplush animals. In one embodiment, a single limiting factor of what typeof hand-held device 12 that may be used is the device 12 includes anoptical detector 30 or other means of reading the image from displaydevice 10, and the technology within device 12 to interpret the receiveddata. The technology for processing the data as described below could beimplemented in hardware, software, or a combination of hardware andsoftware.

The optical detector of the hand-held device acquires the light from thedirection of the computer monitor and filters out the non-usablefrequencies. The hand-held device first processes the received data todetermine what image was present. The band pass filter eliminatesleftover signal noise and leaves any present image frequency. Thus, thesignal leaving the band is either something or nothing. The rectifiertakes the absolute value of the signal and adds it to the integrator.The integrator accumulates each portion of the image signal received ineach refresh to determine if the signal received was an image or noise.If there is sufficient signal to conclude that a signal was present, themicro controller reports a value of one; otherwise, a value of zero isreported.

The hand-held device determines how frequently it must sample themonitor. The signal after its original filtration also proceeds to anenvelope detector, which attempts to match the signal received tovarious monitor refresh rates. The matching results are passed to thevertical sync reconstruction to determine which of the standard refreshrates the monitor is using. The application program thus transmitssufficient information along with the data to the hand-held device sothat the device can determine the horizontal frequency of the computermonitor and determine how many times per second it must look for animage and record data bits.

Referring to block diagram FIG. 20, it may be observed that light isfirst acquired 200, then processed through a high pass filter 210 andlow pass filter 220 to detect the auxiliary data as described above. Afirst portion of the signal is then processed by envelope detector 230and vertical sync reconstruction 240 to determine the configuration ofthe monitor, the results of which are passed to the microcontroller 280.A second portion of the signal travels through the VEIL frequencybandpass filter 250, signal rectifier 260, and integrator 270 pursuantto the repeated requests of microcontroller 280. Microcontroller 280,upon receiving data upon each segment request, provides system output290.

In view of the foregoing description, it will be seen that the severalobjects of the inventive subject matter are achieved and otheradvantages are attained. The embodiments and examples were chosen anddescribed in order to best explain the principles of the inventivesubject matter and the practical applications to thereby enable othersskilled in the art to best utilize the inventive subject matter invarious embodiments and with various modifications as are suited to theparticular use contemplated.

As various modifications could be made in the constructions and methodsherein described and illustrated without departing from the scope of theinventive subject matter, it is intended that all matter contained inthe foregoing description or shown in the accompanying drawings shall beinterpreted as illustrative rather than limiting.

The breadth and scope of the inventive subject matter should not belimited by any of the above-described exemplary embodiments, but shouldbe defined only in accordance with claims of the application and theirequivalents.

The present system and hand-held devices useful in the system may be forthese purposes distributed by various business entities, which mayinclude not only the sponsor of a program, event, or other sponsoredtransmission, but may also include various businesses, services andorganizations having commercial relationships with the sponsor. Viewer'suse of the new devices of the invention accordingly provide commerciallyadvantageous results relating to television advertising, promotions andother sponsored transmissions, wherein new system and devices and theiruse is effective to:

Attract viewer attention to advertising, promotions and other sponsoredtransmissions,

Effectively increase awareness and retention of message and product,

Induce viewers to respond and take action after viewing to sponsoredtransmissions,

Differentiate a sponsor's product or service from those of competitors,

Stimulate viewers to desire to watch advertising, promotions andtransmissions,

Provide a novel, cost-effective, brand efficient, and enjoyable mediumfor enhancing advertising, promotions and other transmissions,

Cause advertising, promotions and transmissions to take on special eventsignificance,

Build customer traffic in places of business or points of saleassociated with sponsors of advertising, promotions and othertransmissions,

Increase consumer loyalty to sponsors of advertising, promotions andtransmissions,

Increase viewer involvement in sponsored advertising, promotions andtransmissions,

Cause viewer retention of sponsor identity and advertising orpromotional content,

Enhance sales volume resulting from the sponsorship of advertising,promotions and transmissions,

Provide lasting value to sponsored advertising, promotions andtransmissions,

Provide enhanced viewer/spectator involvement in sporting and racingevents,

Afford viewers of television programming an opportunity to interactmeaningfully with programming content and with advertising, promotionsand transmissions,

Enhance viewer good-will relative to advertising, promotions and events,and/or

Cause development of still other opportunities for interactive use ofthe devices.

FIG. 21 illustrates another example of the inventive subject matter.Composite signals projected by the display device 10 (shown in FIG. 1)to be received by an interactive hand-held device 2100. The compositesignals may include a video signal, an audio signal, and/or auxiliarydata. The video signal may have the auxiliary data encoded (e.g., addedvia modulation or another technique) in the video signal so that thedevice 2100 can optically receive the video signal and decode (e.g.,demodulate) the auxiliary data from the video signal. Optionally, theaudio signal may have the auxiliary data encoded (e.g., added viamodulation or another technique) in the audio signal so that the device2100 can audibly receive the audio signal and decode (e.g., demodulate)the auxiliary data from the audio signal. Optionally, the video signaland/or the audio signal may not have the auxiliary data encoded in thevideo signal or audio signal, but the device 2100 may detect the videosignal and/or the audio signal (or a portion thereof) and, responsive tothis detection, the device 2100 may obtain the auxiliary data, such asby downloading the auxiliary data from a remote location via one or morewired and/or wireless connections with the remote location (e.g., aserver that is not included in the device 2100).

The device 2100 may be similar or identical to the hand-held device 12,and may be in the form of any type of hand manipulable device such as asmart card, cell phone, PDA, or other palm like device. Hand-held device2100 may include one or more embodiments of the circuitry shown and/ordescribed herein. Hand-held device 2100 may be held in the palm orbetween the fingers of a user in the generally vicinity of displaydevice 10, typically within the same room and, when necessary, orientedso that when a front surface 2116 of the device 2100 faces displaydevice 10, a lens 2136 (e.g., a camera) of the device 2100 can opticallyreceive light from a visual representation of the video signal fromdisplay device 10. The device 2100 includes an audio pick-up device2104, such as a microphone, that can audibly receive audio signals fromthe device 10.

Hand-held device 2100 is depicted in FIG. 21 as a mobile phone and shownin a manner in which it may be held and used. Hand-held device 2100 mayinclude a PDA-sized housing 2114 that encloses various circuits andcircuit components (“circuitry”) of the device. Housing 2114 may be ofany hand-held shape desired that is functional and hand manipulable,examples of which include a PDA, cell phone, PCMCIA card, snapshot-typecamera, replica sports helmet, replica baseball bat or football, orscaled racing car.

Hand-held device 2100 is illustrated to have the pick-up device 2104,the RF antennae 2168, and/or the lens 2136 so that device 2100 mayreceive auxiliary data by optical, audio, and/or electrical (e.g.,wireless) techniques. When hand-held device 2100 includes such receptionmeans, it may be used with various promotional opportunities in variouslocations. Thus, with one example of the present invention, a user mayreceive auxiliary data on hand-held device 2100 at a game inside astadium or arena electrically via RF antennae 2168, and may also receiveauxiliary data while watching a game at home optically by positioningfront surface 2116 toward display device 10. Users may thus participatein promotional in multiple locations using the same hand-held device2100. Optionally, the hand-held device 2100 can audibly receive theauxiliary data encoded in audio of the composite signal and/or thehand-held device 2100 can wirelessly receive the auxiliary data via theantennae 2168 in response to receiving the audio signal (or a portionthereof).

For example, in one aspect, the auxiliary data may be added to anexisting audio signal by modulating the audio signal with the auxiliarydata. The auxiliary data may be added in that the audio signal may becreated or otherwise generated for a program being displayed on thedevice 10 (e.g., a television show, movie, sporting event, etc.) and theauxiliary data is added to this audio signal without being a part of theoriginal audio or part of the sounds of the show, movie, sporting event,etc. Upon receiving the audio signal with the auxiliary data encodedtherein or thereon, the device 2100 can demodulate the auxiliary datafrom the audio signal. The auxiliary data that is encoded into the audioand/or video signal may be referred to as a watermark. The watermark isdata added to the audio signal, video signal, or a composite signal ofthe audio and video signals. The watermark can be used to trigger one ormore devices other than the device 10 (e.g., mobile phones, tabletcomputers, other computers, etc.) to present information to a user, toobtain additional data, or the like, as described herein.

In another example, the auxiliary data may not be included in the videosignal or audio signal presented by the device 10, but may be obtainedby the device 2100 in response to the device 2100 optically and/oraudibly detecting the video signal, the audio signal, or a portionthereof. The device 2100 may detect all or part of the video signaland/or audio signal and compare this detected signal (video and/oraudio) to designated video fingerprints and/or audio fingerprints. Avideo fingerprint includes a video or portion thereof that is uniquerelative to other videos. For example, a video fingerprint may include ashort video (e.g., a few seconds long) that is different from othervideos or other portions of the same video. The device 2100 can comparethe video that is detected from the device 10 and compare the detectedvideo to one or more video fingerprints (stored locally in a memory ofthe device 2100 and/or in a remote memory, such as a server). If thedetected video matches the video fingerprint, then the device 2100 mayobtain the auxiliary data from a remote location. As another example, anaudio fingerprint may include an audio clip that is different from otheraudio signals or other portions of the same audio signal. The device2100 can compare the audio that is detected from the device 10 andcompare the detected audio to one or more audio fingerprints (storedlocally in a memory of the device 2100 and/or in a remote memory, suchas a server). If the detected audio matches the audio fingerprint, thenthe device 2100 may obtain the auxiliary data from a remote location. Inthese situations, the device 2100 obtains the auxiliary data in responseto detecting the video and/or audio signal that matches the video and/oraudio fingerprint.

A typical manner in which hand-held device 2100 is used to receiveauxiliary data is as follows: A user watches a television presentationor listens to an audio presentation such as a commercial, televisionprogram, movie, or other program. The video signal and/or audio signalis detected and received by the lens 2136 and/or pick-up device 2104.The video signal can include a video signal modulated with auxiliarydata, the audio signal can include an audio signal modulated with theauxiliary data, and/or the video and/or audio signal may not bemodulated with the auxiliary data. The device 2100 may then demodulatethe auxiliary data from the received signal and/or obtain the auxiliarydata in response to detecting an audio and/or video fingerprint. Theauxiliary data may then be used in accordance with one or more examplesdescribed herein. Optionally, the auxiliary data may be used to providea user with an enhanced interactive experience. For example, theauxiliary data may be displayed on the device 2100 to the user and mayprovide additional information about the program being shown on thedevice 10, the sporting event in the vicinity of the user or shown onthe device 10, or the like. The auxiliary data may provide or be used toprovide a game on the device 2100 that is played along with the videosignal and/or audio signal being presented via the device 10. A display2144 of the device 2100 may be similar to LCD 44 or may be another typeof display device, and can present the auxiliary data as describedherein.

The device 2100 may include one or more of the microprocessors describedabove for performing the operations described herein. Themicroprocessors may receive signals from the device 10, such as atelevision, where the signals include an audio signal. Themicroprocessors may receive such a signal via the pick-up device 2104.The antenna 2168 can wirelessly receive or otherwise obtain data inresponse to receiving the signal with the audio signal from thetelevision. For example, the antenna 2168 can be similar to the antenna68 and can wirelessly communicate with a remote location (e.g., one ormore servers) via the Internet and/or another network to obtain thedata. The display 2144 can be similar to the LCD 44 or other displaydevice, and can visually display a user input screen based at least inpart on the data that is wirelessly received. For example, themicroprocessors can direct the display 2144 to show a game, images,videos (which may be formed from two or more images), or the like, inconjunction with a program being shown on the device 10.

The microprocessors can provide a game that the user can play on thedevice 2100, with the game being interactive with the show, program,sporting event, etc., being displayed on the device 10. A user can watcha television show and the device 2100 can detect audio (and/or video)from the television show, with the audio and/or video including theauxiliary data and/or the device 2100 obtaining the auxiliary dataresponsive to detecting the audio and/or video of the television show.The device 2100 can then use the auxiliary data to present a game to theuser, with the game including trivia questions, challenges, or the like,that are in the television show, based on events in the television show,or otherwise related to the television show. Optionally, the device 2100can present additional information about characters and/or actors in thetelevision show, trivia about the television show, statistics forathletes shown in the television show (e.g., a sporting event),promotional information about sponsors of the television show, etc.

The device 2100 can receive user input from the user in response to thedisplay shown to the user on the device 2100 and can process this userinput. For example, the user may select answers to the game being shownon the device 2100, the user may move a video game character in a gamebeing shown on the device 2100, the user may type and/or select text orimages shown on the device 2100, or the like, as user input. The device2100 can then examine the user input in connection with the game toassist the user in playing the game. For example, the device 2100 candetermine if the user input represents a correct or incorrect answer toa question in the game, if the user input moves a character in one ormore designated ways (to advance in the game), if the user inputrequests additional or different information than the information beingpresented to the user. The device 2100 can then advance play of the gameand/or otherwise change the information being shown to the user on thedevice 2100 based on the user input that is received. For example,responsive to receiving user input, the device 2100 can communicate withone or more remote locations (e.g., servers) to obtain additional data.This additional data may then be used to change the display shown on thedevice 2100, such as by showing the results of the game, additional gameplay, different information requested by the user input, etc.

The information presented by the device 2100 in connection with theinformation being shown on the display 10 can provide a second screenexperience to a user of the device 2100. This experience may provide theuser with additional video and/or audio that is not concurrently beingshown on the display 10, but that may be related to the program beingshown on the display 10. For example, the additional video and/or audiomay be additional or deleted scenes from a movie or show, highlights ofother sporting events, or the like. As another example, the device 2100may provide web browsing software in response to receiving the auxiliarydata. For example, the auxiliary data may include a link to a web pagethat is automatically opened by web browsing software operating on thedevice 2100.

In view of the foregoing description of the inventive subject matter andpractical embodiments it will be seen that the several objects of theinventive subject matter are achieved and other advantages are attained.The embodiments and examples were chosen and described in order to bestexplain the principles of the inventive subject matter and its practicalapplication to thereby enable others skilled in the art to best utilizethe inventive subject matter in various embodiments and with variousmodifications as are suited to the particular use contemplated.

As various modifications could be made in the constructions and methodsherein described and illustrated without departing from the scope of theinventive subject matter, it is intended that all matter contained inthe foregoing description or shown in the accompanying drawings shall beinterpreted as illustrative rather than limiting.

The breadth and scope of the inventive subject matter should not belimited by any of the above-described embodiments, but should be definedonly in accordance with the following claims and their equivalents.

The invention claimed is:
 1. A decoder device comprising: one or moremicroprocessors configured to receive a composite signal from atelevision, the composite signal having auxiliary data encoded into anaudio signal of the composite signal, the one or more microprocessorsconfigured to extract the auxiliary data from the audio signal in orderto trigger performance of one or more actions on a mobile phone thatincludes the one or more microprocessors.
 2. The decoder device of claim1, wherein the one or more microprocessors are configured to extract theauxiliary data that is modulated into the audio signal.
 3. The decoderdevice of claim 1, wherein the one or more microprocessors also areconfigured to extract the auxiliary data from a video signal of thecomposite signal.
 4. The decoder device of claim 1, wherein thecomposite signal is a television signal and the one or moremicroprocessors are configured to extract the auxiliary data from thetelevision signal.
 5. The decoder device of claim 1, wherein the one ormore microprocessors are configured to visually display a user inputscreen on the mobile phone based at least in part on the auxiliary dataresponsive to extracting the auxiliary data from the composite signal.6. A method comprising: encoding auxiliary data into an audio signal ofa composite television signal that includes the audio signal and a videosignal, wherein the auxiliary data is encoded into the audio signal andincludes information that directs one or more mobile phones that receivethe audio signal and that extract the auxiliary data from the audiosignal to perform one or more actions on the one or more mobile phones;and communicating the composite television signal to one or more displaydevices for presentation to one or more users.
 7. The method of claim 6,wherein the auxiliary data is encoded into the audio signal bymodulating the auxiliary data into the audio signal.
 8. The method ofclaim 6, wherein the auxiliary data is encoded into the video signal ofthe composite signal.
 9. The method of claim 6, wherein the compositesignal is a television signal and communicating the composite signalincludes communicating the television signal for presentation on one ormore televisions.
 10. The method of claim 6, wherein the auxiliary datathat is encoded into the audio signal is a trigger that, responsive tobeing received by the one or more mobile devices, directs the one ormore mobile devices to visually display a user input screen based atleast in part on the auxiliary data.
 11. A tangible and non-transitorycomputer readable medium comprising instructions that direct one or moremicroprocessors to: detect receipt of an audio signal in a compositetelevision signal at a mobile phone; extract auxiliary data that isencoded in the audio signal; and perform one or more actions on themobile phone responsive to detecting the audio signal and using theauxiliary data that is extracted.
 12. The tangible and non-transitorycomputer readable medium of claim 11, wherein the instructions directthe one or more microprocessors to extract the auxiliary data that ismodulated into the audio signal.
 13. The tangible and non-transitorycomputer readable medium of claim 11, wherein the instructions directthe one or more microprocessors to also extract the auxiliary data froma video signal of the composite television signal.
 14. The tangible andnon-transitory computer readable medium of claim 11, wherein theinstructions direct the one or more microprocessors to visually displaya user input screen on the mobile phone based at least in part on theauxiliary data.